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Engagement and Improve Learning in an Undergraduate Environmental 
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Abstract 

Learning outcomes for introductory college-level science classes include content knowledge and a range of 
critical thinking and analysis skills. In this context, rich writing assignments that engage students in 
researching content, constructing arguments, and critiquing other students’ work are highly desirable but 
unwieldy due to large enrollment. Calibrated Peer Review (CPR) is a web-based program that engages peers 
in writing and in reviewing each other’s writing. The authors explored whether students saw CPR as simply a 
way to improve writing skills, or if they also thought the process had other educational benefits. The 
researchers gathered student perspectives using open-ended qualitative questionnaires. We report on the 
themes that emerged in these survey responses over two semesters, and examine some implications of the 
students’ perspectives for future use of CPR in an introductory science course. 
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Abstract 

Learning outcomes for introductory college-level science classes include content knowledge 
and a range of critical thinking and analysis skills. In this context, rich writing assignments 
that engage students in researching content, constructing arguments, and critiquing other 
students' work are highly desirable but unwieldy due to large enrollment. Calibrated Peer 
Review (CPR) is a web-based program that engages peers in writing and in reviewing each 
other's writing. The authors explored whether students saw CPR as simply a way to improve 
writing skills, or if they also thought the process had other educational benefits. The 
researchers gathered student perspectives using open-ended qualitative questionnaires. We 
report on the themes that emerged in these survey responses over two semesters, and 
examine some implications of the students' perspectives for future use of CPR in an 
introductory science course. 

Keywords: peer review, science education, instructional technology, writing 


Introduction 

Learning outcomes for introductory college-level science classes include content knowledge 
and a range of critical thinking and analysis skills (e.g., NRC, 2003), and may include 
written communication of scientific issues or concepts. Because simple lecture and 
memorization is not an effective mode of learning for many students (e.g., Bransford et al., 
1999; Mayer, 2009), and does not address several common learning outcomes, instructors 
are increasingly experimenting with teaching approaches that use active engagement 
(Handelsman et al., 2004; Judson & Sawada, 2006; Peleaz, 2002; Prince & Felder, 2007). 

In this context, rich writing assignments that engage students in researching content, 
constructing arguments, and critiquing other students' work are highly desirable. However, 
in large enrollment introductory science classes such assignments have traditionally been 
avoided because of the enormous amount of instructor time and effort involved in critiquing 
the assignments and providing meaningful feedback. 

Calibrated Peer Review (CPR) is a web-based program that engages peers in writing and in 
reviewing each other's writing (Clase, Gundlach, & Palaez, 2010; Rudd, Greeenbowe, & 
Hand, 2007). CPR is designed to allow for rich writing and feedback experiences with far 
less instructor time. As users of CPR in an introductory environmental science class, and 
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given the relatively small research base on CPR effectiveness and student viewpoints (e.g., 
Fosmire, 2010; Gunersel & Simpson, 2009), the authors qualitatively examined students' 
perspectives on CPR, in particular what benefits and limitations they felt were involved in 
having CPR as a significant part of their course. Specifically, the authors were interested in 
whether students saw CPR as simply a way to improve writing skills, or if they also thought 
the process had other educational benefits. Thus, after a series of CPR assignments, the 
instructor surveyed the class using open-ended questions to gather student perspectives. 
Here we report on the themes that emerged in these survey responses over two semesters, 
and examine some implications of the students' perspectives for future use of CPR in an 
introductory science course. 


Calibrated Peer Review 

Peer review is a hallmark of the publication process for scientific research, yet it is rarely 
used in the typical college classroom (Clase, Gundlach, & Paleaz, 2010; Prichard, 2005). 

The premise of using peer review as an instructional strategy for written work is that it 
helps students hone their writing skills through evaluating other students' essays in a 
guided process, receiving feedback from peers, and comparing other essays to their own 
(Fosmire, 2010; Prichard, 2005). In science, peer review is an authentic component of the 
scientific process, and writing about concepts and constructing arguments and analyses 
have additional advantages because they require a deeper understanding of material and 
force students to explain and justify their conceptions. Calibrated Peer Review (CPR) is an 
integrated online system that allows students to submit and evaluate writing assignments 
online, and includes a calibration phase to ensure that students are reviewing written work 
in a consistent fashion (Gunersel, Simpson, Aufderheide, & Wang, 2008). The instructor 
begins the process by introducing the assignment, and the remainder of the process is 
entirely online (Prichard, 2005). Students perform guided background reading, respond to a 
writing prompt, use a rubric to review examples of answers to the prompt (in the calibration 
phase; these example answers have been prepared by the instructor), and then review the 
work of three other students and their own work using the same rubric. The review process 
is double blind, and only the instructor knows the identity of the author and the reviewers 
for each written response. At the end of the process each student receives scores and 
feedback on their work from three reviewers and each student also sees how their review of 
the example answers and other students' work compares to other reviewers. 

Existing research on CPR has focused largely on quantitative measures of student 
improvement (e.g., Berry, Carlson, & Millard, 2006; Furman & Robinson, 2003; Pelaez, 
2002). However, Gunersel & Simpson (2009) used qualitative methods to understand 
faculty views of CPR while Keeney-Kennicutt et al. (2008) conducted a mixed-methods 
study on student perceptions of CPR over the course of 7 semesters in an introductory 
chemistry class and found that most students did not like CPR initially but grew to 
appreciate it. Complementing this prior research, the work reported here was designed to 
examine how students perceive Calibrated Peer Review (CPR) as a learning strategy in an 
introductory environmental science class. This work was undertaken as formative 
assessment to improve the use of CPR in the class in future semesters, and to provide 
insight that might be of value to others using CPR or similar approaches in introductory 
science teaching. 
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Methods 

Participants in this study were undergraduate students from two semesters in Introduction 
to Environmental Science at a large, co-educational, Midwestern, state university. The 
students were taking a large-enrollment, freshman-level environmental science class either 
as an elective to meet a general science requirement, or as a required class for an 
environmental major. The average age of the students was 20 years old and there was a 
relatively even mix of males and females (46% female). A total of 46 students from two 
semesters responded to the in-class survey. The course covered a range of environmental 
themes and issues, and included an emphasis on understanding how scientific knowledge 
and evidence is used in real-word decision-making. During a 16-week semester the 
students had four CPR assignments: an initial assignment to learn how to use the system 
("why did you choose your major"), and three assignments tied to major themes of the 
course: Environmental Considerations for Alternative Energy; C0 2 the Greenhouse Effect 
and Global Warming; Earthquakes and Plate Boundaries. 

The research team used a survey approach to the research question and surveyed students 
at the close of the semester. The department approval for this study was obtained prior to 
the launch of the survey. As an experienced teacher who had used peer-reviewed writing 
assignments in an introductory science class several times, but who was a first time user of 
CPR, the instructor was interested in learning more about students' views on CPR. The 
instructor designed a series of open-ended questions designed to elicit student's views 
about CPR. The survey consisted of three questions: 

• The first question was designed by the instructor to encourage students to 
reflect on the learning goals associated with the use of CPR in the course: 
"What do you think are the benefits of having CPR as a part of this class (how 
does it contribute to learning goals)?" 

• The second question was designed by the instructor to gather insight in to 
concerns students had with CPR and ways CPR or its specific use in the class 
could be improved. "In what ways would you suggest that we 
change/improve CPR for use in this class next time?" 

• The third question was designed by the instructor to give students the chance 
to record any other thoughts they had on CPR as used in this class. "Any 
other comments you would like to make about CPR." 

The survey was administered during class time and each student who turned in a survey was 
given extra class participation credit with non-participating students (n = 2) not being 
penalized. The class was given 15 minutes to complete the survey. After collecting the 
surveys from the students each was stripped of personal data by the department secretary 
and entered into a table for data analysis. Data stripping included removing the name of the 
student and any personal details such as paper topics and personal remarks about grades. 
We then employed open coding to try and find common themes among all three research 
questions. Common themes were developed from the data as well as the current literature 
on peer review and prior research on Calibrated Peer Review. From open coding we were 
able to find approximately 34 themes that we used to create six categories supported with 
student comments. As we analyzed the data we allowed themes and categories to emerge 
from it rather than predetermining them (Lincoln and Guba, 1985). 
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Findings 

There are six categories that arose during the research process. Each category was derived 
from the survey responses, supported by direct quotations from the survey participants, and 
formed into assertions. Each assertion is a categorically supported excerpt within the overall 
picture that forms calibrated peer review in the science college classroom. 


1. Grading and Feedback 

Within CPR's platform students are expected to compose an essay as well as evaluate three 
other students essays. The evaluations contribute to the students' grade who composed the 
essay. Some students were satisfied with how CPR's grading and evaluation process 
operates whereas some students believed that the grading and evaluation process of CPR 
had a negative impact on grades. One student said "CPR allows me to review how other 
explain the same topic as me, which allows me to get multiple views on the same subject." 
Another student said that CPR " may also help the student learn to critically evaluate the 
work of others." One of the displeased students said, "the peer review sometimes brings 
your grade down even though you did well." A second student said, " some graders don't 
thoroughly read" which implies that he/she received a grade below what he/she expected to 
receive. CPR incorporates students into the grading process which overall seems positive 
however it does have the potential to negatively impact a students grade even if the essay 
is perceived as practically perfect. 

Calibrated Peer Review (CPR) allows students to drive the grading and feedback process 
with class writing assignments. Through calibration of instructor provided material and 
grading rubrics students grade each other's work. Each student's work is reviewed by three 
of his or her peers and the score is calculated. Feedback is given in the form of comments 
on specific rubric items and grading is done through a point system. Students who have 
experienced this system have mixed feelings about its reliability. 

We found through axial coding and strength rating that students experience two distinct 
situations, as shown above, in regards to CPR grading and feedback. The first situation is 
where the surveyed student is happy with the experience and feels that getting multiple 
peer reviews helped them with their own grade. One student said, "The main benefit from 
this (CPR) is seeing what other students mess up on, and learning from their mistakes . . ." 
Another student felt that, "reading the feedback of my peers on my writing has given me 
confidence that I can get my points across effectively in writing." Students who commented 
on the first situation, being happy with the experience and getting multiple peer reviews, all 
focused on the process of "students seeing what other students are writing and allowing to 
have the opportunity to grade others and practice reviewing skills". One student sums up 
this practice by stating, "When editing other's work you see your own in a different light." 

The second situation that kept appearing during coding and analysis revolved around 
students who were not happy with their grades or the process for acquiring feedback. One 
student commented, "Some grader's don't thoroughly read; the numerical score is 
somewhat arbitrary." Through strength rating we were able to identify that students felt 
very strongly about other students not controlling their grades. Comments ranging from 
"less weight should be given to student grading" to "I know if a professor had graded my 
work I would have secured full marks" students felt that other students did not have the 
proper training or were not taking the time necessary to grade fairly. 
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2. External Factors Controlling CPR 

A course instructor has considerable latitude in using CPR, including controlling the number 
of CPR assignments and how they are integrated in to the course, as well as the CPR writing 
prompt, reading material, timing, grading rubric and score weighting. 

Many comments on external CPR were specific and conveyed a strong opinion. 

A frequent issue mentioned was the number and depth of 'guiding questions' for the writing 
prompt, and how these related to the evaluation rubric. This ranged from "more questions 
should be asked that are simpler to answer" to "If there were fewer, but more in-depth 
questions it would be better" and "Let us write about what we want or learned from the 
articles instead of giving us questions to answer". One student wrote "I thought sometimes 
the grading questions were fairly rigid while the text guidelines were very general" which 
echoed another student's comment that "I feel like the points we should focus on writing 
about should be more clearly indicated in the writing prompt." Similarly "I think that we 
should be given all the questions we need to answer when we write it because when we go 
to calibration the questions needed for our writing is different." 

Another common feeling in the responses was of frustration with rubric questions that had a 
subjective component, for example "sometimes they tend to be vague and it makes it hard 
to know if you should give points to someone or not". 

Several students commented on the way CPR was integrated in to the class. One 
suggestion was to "consider ... introducing the topic in class and quickly explain how it fits in 
to our current topic of discussion" and "if we discussed the CPR topic in class ... prior 
submission of the CPR assignment, then the students may have a better understanding of 
what the CPR essay should entail." 

The number of assignments and the timing of the assignments and their components was 
also a focus of comments: "I think starting assignments earlier and adding one more 
assignment would be beneficial" and "You could have a CPR assignment due every other 
week or after each new chapter". Within the assignments timing was an issue for some 
students: "I would suggest allowing more time to complete the text as well as more time 
between the text deadline and the reviews and calibrations" and "More time regarding the 
calibration part". Similarly students suggested, "Put more time between text submit and 
review" and "Make the writing periods two weeks and the calibration and review time 1-2 
weeks". 

These comments on student perceptions of the elements of CPR usage controlled by the 
instructor are helpful in the instructor's consideration of how to change use of CPR in future 
sections of this course. Some comments indicate to the instructor that more reinforcement 
of the purpose of CPR within the course is needed, as well as more discussion within the 
class of the nature of evaluation and grading. The instructor also noted the need for better 
integration of CPR topics in to the flow of course content. CPR allows the instructor to 
control significant parts of the learning experience. To set up a CPR assignment the 
instructor writes three calibration examples, prepares the grading rubric, sets the deadlines, 
and assigns the point system. 

Outside of the above-mentioned items there are also classroom specific processes that 
affect CPR. We found that students focused on preparation, time, due dates, number of 
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assignments, length of assignments, and format of the assignment. For preparation students 
demonstrated that they thought CPR assignments should be discussed in class prior to 
assignment. One student commented that, "if we discussed the CPR topic in class prior to 
submission of the CPR assignment, then the students may have a better understanding of 
what the CPR essay should entail." Others commented on using class time to "tell us about 
the assignment" and "announce CPR assignments earlier" as well as "introducing the topic in 
class and quickly explain how it fits into our current topic of discussion". 

The amount of time given to complete the CPR assignment and the due dates associated 
with each assignment dovetailed in most student comments. We found that students 
wanted "more time to complete assignments" in conjunction with "concrete due dates. 

Other students commented that "knowing the CPR assignments and due dates in advance, 
like on the first day of class" would allow them to be more prepared. Across the board 
students felt that "more time" for activities like text entry, calibration, and reviewing would 
improve the experience. 

The number, length, and format of the assignments also rated very strongly in strength 
testing and axial coding. We found that students did not have a specific complaint or 
compliment but tended to be from one end of the spectrum to the other about the CPR 
assignments. One student commented, "make smaller assignments rather than bigger ones", 
while another student commented, "make topics more open". Other students commented, 
"make 5 peer reviews and less calibration", while another commented, "make sure everyone 
gets their paper graded/reviewed 3 times." The format of the CPR assignments was also up 
for debate among the students. While some felt that they were too restrictive, commenting, 
"let us write about what we want or learned from the articles instead of giving us questions 
to answer," other students felt that the guiding questions helped them structure their work 
but thought that, "the focus questions should be less numerous" and "putting 6 to 8 
questions into an essay ... is challenging, unnatural and may force someone to promote a 
viewpoint they otherwise would not." 

While the student responses to external features of CPR were varied they were all strong on 
the strength rating scale as very few students wrote neutrally about the process. 

3. Internal Factors Regarding CPR 

The basic structure of the CPR program is fixed, and parts of this structure were viewed by 
students as having an impact on their experience. Several students pointed out limitations 
with the text formatting process which is similar to html coding: "the text entry seemed a 
little cumbersome, perhaps it could be changed to make it more user friendly." and "I would 
suggest that more instruction be given on how to format the text". A student who never 
worked out how to format text correctly was frustrated that "No matter how it looks when I 
submit it, I always see one huge paragraph when I do my personal review". 

Another aspect of CPR that is fixed is that students have to complete three calibration 
reviews of answers written by the instructor, and then three reviews of the work of their 
peers followed by a self-evaluation. The calibration portion was viewed by one student as 
being too time consuming, which led to the suggestion that "instead of three calibrated 
reviews maybe have two instead". Another student was frustrated when other students did 
not complete their reviews "Make sure everyone gets their paper graded/reviewed all 3 
times even if that means the teacher does it". Another theme was a concern about how a 
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student was supposed to learn from the calibration "I didn't care for how calibrations went 
because your style of grading may not be on par with CPR's and you miss points solely for 
the fact you aren't trained". 

There were also comments on the value of this structure, such as "During the calibrations 
section of three peer articles, I was able to develop a critiquing skill, so I can assign each 
article a grade without any bias." Various aspects of the structure of CPR can alter learning 
experiences. Calibrated Peer Review, the web-based system used in this class, was 
developed for another university and licensed out for others to utilize in their instruction. 

The hard coding of some of the features was a concern for the students surveyed. Hard 
coded features such as; text formatting was noted to have impact on the student learning 
experience. 

The hard coded facet of CPR that students focused on was the text formatting that the 
system imposed during their writing. One student commented, "it's hard to format it the 
way you would like, mainly with several paragraphs. No matter how it looks when I submit 
it, I always see one huge paragraph when I do my personal review." Other students 
piggybacked off of this statement by adding that they "really don't like how the text needs 
to be formatted" and "the text entry seemed a little cumbersome." While we agreed that 
these issues were real and important to the students, it is important to note that as of now 
the hard coded aspects of CPR cannot be changed by our university. Text formatting is set 
for the duration as well as the number of calibrations the system requires to make sure the 
student is an adequate reviewer. Such hard coded features of CPR are present to ensure a 
uniform experience from start to finish- students complete three calibrations and after that 
complete three peer reviews. 

4. Connection to the Class 

CPR gives science students an opportunity to learn course material in unique way. This 
unconventional style of learning impacts how the material is learned. CPR forces students to 
further investigate information on a topic outside of class related materials. One student 
said that (CPR) "helps you get involved in the material you are learning about in class at the 
time" which is evidence of the alternate teaching method CPR has to offer. Another student 
sates "it(CPR) allows students a different way to learn topics covered in class." CPR has to 
ability to help a student learn and understand course material better. There are numerous 
personal statements by students that support this claim. One student stated that "it (CPR) 
really helps me learn and remember a topic if I put it in presentation, or teachable, form." 
Another student with a similar thought said "I feel like I do have a better understanding of 
the topics covered in the CPR assignment after I finish them." One of the reasons for this 
better understanding is the fact that the students were forced to further research a topic 
before writing on the topic, which is expressed in student comments. One student "it (CPR) 
gives us a chance to research outside of the book." A second student said that "(CPR) 

Forces student to research materials more in-depth" which has the obvious consequence of 
that specific topic being better remembered as well as understood. 

CPR affects how a student learns that material; most responses revealed that CPR has a 
positive result on recall of class material. CPR requires students to read additional material 
related to class content, and to use that content to address a question posed in a writing 
prompt. The student then reads and evaluates six additional written responses to the same 
prompt, which gives extensive exposure to the material and different ways of describing and 
using it. The main aspect of this was described succinctly by one student "the amount of 
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repetition involved in critiquing other student's papers allows students to have an excellent 
understanding of the subject matter". Similarly "By reading other's work it reinforces the 
learning material" and "It helps us to learn what we need to write by looking at what others 
write in response to the prompt". One student suggested, "CPR is better than a term paper. 
It does make us think more about our class work, which would otherwise be mindless 
memorization and repetition. It basically makes us learn the material." Another student 
summed it up "by reading, writing and reviewing, we learn more and have it stay longer." 
Other student comments in this theme are that CPR "helps students learn the material the 
same way writing any paper does", for example "CPR assignments made me read material 
and make sense of it, enough so I could write an essay about the topic" and "It also takes a 
higher level of understanding to write a paper about something (than) it does to answer 
multiple choice questions". Similarly "It helps to gather a better understanding of the 
material being studied for class because we are forced to obtain information for the essays." 
The comments overall indicate that many students recognize the impact that a written 
assignment and reading multiple responses to this has on their familiarity with and 
understanding of that area of the course material. 

5. Enhancing Basic Research and Reading Skills 

Prior to writing an essay it is usually a good idea to do some type of research or gathering 
of information. High school students and undergraduates alike loosely refer to this type of 
information gathering as research. It is by no means the same type of research that a 
professor does, which is the creation of unique data. This allows a student to write an 
intelligent and coherent composition about a specific topic. 

CPR assignments are composed of multiple pieces, with one of them being to compose an 
essay on a specific topic. If a student wants to do well on the composition section then 
he/she must gather information about the topic. In short CPR forces students to practice 
background research which has an impact on understanding on that topic being studied. 

One students' comment states "some benefits (of CPR) would be researching the topics so 
you are learning more about them." Another students' comment states that CPR allows for 
"good research technique for students." This student said that CPR "allow(s) students to 
improve upon research skills." When students are assigned CPR assignments they gaining 
more than an in-depth knowledge about particular topic. Students who have the privilege of 
using CPR also practice skills that can useful later in life such as doing background research. 

In CPR the student is provided with links to background material that is helpful in 
addressing the writing prompt. This encourages the student to read and review background 
material as the basis for writing about the theme (a process students refer to as "research"). 
For example, one student notes that CPR "Forces students to research materials more in- 
depth" and others note that it "gives you reason to research a subject" and "it gives us a 
chance to research outside of the book." Other students note that this is a "Good research 
technique for students" and "Allows student to improve upon research skills". In CPR each 
student is required to read six responses to the writing prompt carefully enough to be able 
to evaluate it relative to a grading rubric, which gives them practice in reading focused on 
finding particular information or traits in a piece of written material. However few students 
commented on reading skills beyond the simple level, such as "It helps with reading" or "I 
liked being able to read other student's work". Other comments about reading were implied 
in comments about various aspects of CPR, such as "it helps students see what other 
students are writing, and it allows them to have the opportunity to ... practice their 
reviewing skills." 
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6. Writing Skills 

CPR impacts students' writing skills. While the explicit purpose of CPR is not to improve 
students writing skills many students surveyed mentioned it as a tool that impacted their 
writing skills either positively or negatively. One student commented that CPR "allowed me 
to improve my writing skills" but to counter that another student wrote that, "To me this 
exercise does not help me learn or enhance writing skills." Other students commented that 
CPR "helps me to understand and write a good essay" adding that "the benefits are that I 
am much more critical of my own writing." After analyzing and strength testing the 
comments about the writing process and CPR we found that students were split in how 
writing is approached through CPR. For every student that wrote positively, "CPR . . . gives 
me an opportunity to practice my scientific writing skills" there were students that wrote 
negatively, "horrible experience- write the cliche response and get full marks. Why teach 
students to write like this?" Unable to come to a consensus on exactly what kind of impact 
CPR has on student writing we decided to focus on the impact itself finding that students 
ran the gamut from "totally unmemorable writing experience" to "great tool to help improve 
upon writing." 


Discussion 

Prior research on peer review (e.g., Enders, Jenkins, & Hoverman, 2011; Pharo & De Salas, 
2009) has demonstrated that efficacy in content knowledge increases when peer review is 
implemented. This study has added to the existing literature by exploring student's 
perceptions on using CPR in an introductory science class. The findings demonstrated the 
different ways in which students perceive CPR to help or hinder the educational process. 
Other research in peer review has demonstrated that perception of student peer review has 
both positive and negative connotations (Mulder, Eiger, & Brady, 2012; Spndergaard, & 
Mulder, 2012). Derived directly from the student responses, the most frequently mentioned 
aspect of CPR, was that peer reviewing helped them get a better understanding of their own 
work as well as understand the course material more broadly. This finding resonates with 
other literature in the field (e.g., Enders, Jenkins, & Hoverman, 2011) and speaks to the 
nature of experiential learning that peer review provides students. Research on peer review 
has indicated negative response to students grading students (e.g., Brill & Hodges, 2011) 
and CPR was not without its detractors; the student's statements about the number of 
calibrations as well as the grading subjectivity ranked highest for negative aspects. 

Students noted that CPR allowed them a chance to see other people's mistakes and 
achievements, allowing them a broader view of the class material. The evaluation section of 
the program allowed them to practice peer review and use their own experiences to drive 
the feedback process. While some students responded positively to this opportunity, others 
felt that it led to subjective grading with peers that were not equipped to adequately grade 
their work. Aside from grading students enjoyed the feedback experience, similar to prior 
research findings (e.g., Spndergaard, & Mulder, 2012), because it allowed them to get a 
peer's perspective on their writing and see if their understanding of a topic was in line with 
the rest of the class. 

There are only so many factors that can be controlled by the instructor when using CPR. 
Students were equally pleased and frustrated by their perceived limitations when it came to 
formatting their assignments and completing the calibrations. Prior research has indicated 
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that the format of peer review software is contentious (e.g., Li, Lui, & Steckleberg, 2009), 
however formatting and the number of calibrations required are hard coded into the main 
program and cannot be changed by the instructor. While some students felt that the 
number of calibrations was too many and took too much time others felt that it let them 
practice their skills before actually grading. The text formatting available in CPR is not 
WYSIWYG (what you see is what you get) so students were not able to format their work 
using return and tab keys- this caused frustration with the program. On the flip side there 
are aspects of CPR that can be controlled by the instructor such as the number of 
assignments and the length of time given for each assignment. Once again there was not a 
consensus of opinion but an equal spread of students that expressed that the assignments 
were too long, too short, too many, not enough, or timed correctly and incorrectly. 

Creating a connection to material learned in class, CPR acted as reinforcement and an 
impetus for gaining a deeper understanding of class content. This finding is in line with prior 
research on peer review which indicates that peer review serves as experiential learning 
giving students a chance to reflect on their own work as they review others work (Brill & 
Hodges, 2011; Mulder, Eiger, & Brady, 2012). Students frequently commented on the fact 
that they felt that the CPR assignment topics were ones in which they invested more time 
and effort in learning and also got to read more about. By reading more about these topics 
from a broader range of literature, including peer writings, they were able to create a 
clearer connection to the material and understand it on a deeper level. Connected to this 
idea is another focusing on enhancing reading and research skills. Students stated that CPR 
assignments allowed them to practice their 'research skills' when it came to preparing for 
and writing the assignments. Vickerman (2009) found similar results when assessing 
student perspectives on peer review; learning is deepened when the student rather than the 
teacher initiates practice. While their definition of research is not based on creating new 
ideas but rather reviewing work all ready in existence and using that to write their papers 
they did strongly assert that CPR fostered a need to be well informed about their topics 
prior to writing. 

CPR also affected writing. While we cannot make the argument that CPR made student 
writing better, but students commented on how their writing has changed over the course of 
CPR assignments. While this may be a product of 'practice makes perfect' the students see 
it as being directly related to the assignments. This finding is reflected in prior research 
(e.g., Likkel, 2012) where students gain confidence in their writing by reviewing others work. 
Contrary to many student comments and prior research, there were some dissenters that 
commented that CPR did not help their writing at all and in some cases made them write in 
ways to achieve a certain grade on the assignment. 

In conclusion, this study revealed several aspects of student experiences and perceptions on 
using CPR to give and receive feedback. Prior research on peer review (e.g., Brill & Hodges, 
2011; Mulder, Eiger, & Brady, 2012; Spndergaard, & Mulder, 2012) and feedback (e.g., 
Vickerman, 2009; Watson & Ishiyama, 2011) informed the current study. Findings provided 
information on how students perceive the usefulness of CPR, how CPR affected his/her 
writing, and how it enhanced their knowledge of the class material. Research skills, the set¬ 
up of the CPR system including calibration and typeset, as well as the chance to compare 
their own work to that of their classmates were also discussed. Future studies may further 
this investigation by interviewing students as they progress through a class using CPR and 
gaining insights into their experiences as it occurs. 
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Significance 

Peer review is a practice used widely in science to validate and critique new research, yet it 
is not always used in the college classroom (Mulder, Eiger, & Brady, 2012; Prichard, 2005). 
Introductory science college classes can range in size from tens to hundreds of students 
with as little as one instructor. Calibrated Peer Review (CPR) allows instructors to assign 
written assignments that permit students to analyze and synthesize information discussed in 
class and practice peer review (Russell, 2001; Watson & Ishiyama, 2011). This study details 
how students perceive the use of CPR in a qualitative fashion, discussing the implications of 
the usage over the course of a semester. 

The implications of the student perspective for using CPR beyond this class are multifaceted. 
In one instance, peer review is supported by students for it's ability to provide experiential 
learning opportunities such as using others work to measure personal understanding of the 
class content. In another instance, the practicalities of students grading each other's work 
without credentials makes the feedback less authentic to some students. The practice of 
review and feedback allows students to practice research skills and encourages reflective 
practice beyond classwork. Student perspectives on using CPR in an introductory science 
class can further the practice of peer review and feedback as a supplement to increase 
scientific writing opportunities without increasing instructor workload. 
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